fix: restore Jackson 2.x property order in RqueueRedisSerializer to prevent stale processing-queue entries after 3.x → 4.x upgrade#300
Conversation
…revent stale processing-queue entries after 3.x → 4.x upgrade
| api "org.apache.commons:commons-collections4:${apacheCommonCollectionVerion}" | ||
| // https://mvnrepository.com/artifact/io.micrometer/micrometer-core | ||
| api "io.micrometer:micrometer-core:${microMeterVersion}" | ||
| testImplementation "com.fasterxml.jackson.core:jackson-databind:2.21.2" |
There was a problem hiding this comment.
Can you please bump the version in root to RC10?
|
Note that any 4.x deployment that has messages currently sitting in the processing queue should drain that queue before deploying this fix. |
|
Can we add this as a prop? |
|
A property based solution seems to get rather messy. I'll try to implement a more robust solution (e.g. making the processing-queue lookups in |
This would be quick change right? We can add a property rqueue.serialization.property.order=ALPHABETICALLY|DEFAULT. |
|
Maybe using |
Yeah this make sense, Also we can take a approach used by Lombok. Serialization format jackson2.x or jackson3+. Both of these would work, but adding legacy does not tell us why this is called legacy. |
…N field ordering Introduces RqueueRedisSerializer.PropertyOrder enum (ALPHABETICAL | DECLARATION) and wires it via rqueue.serialization.property.order (default: ALPHABETICAL). ALPHABETICAL uses Jackson 3.x alphabetical ordering, the native default for RQueue 4.x deployments. No configuration change required for new installs. DECLARATION uses declaration order, matching the Jackson 2.x behaviour of RQueue 3.x. Set this when upgrading from 3.x with messages still in Redis queues, as switching while messages are in-flight causes unexpected retries. The setting is applied in RqueueListenerBaseConfig before any Redis template is created (overriding RedisUtils providers when DECLARATION is requested), and flows through RqueueConfig to RqueueInternalPubSubChannel so all serialiser instances in the application use the same order. Docs: configuration.md and migrations.md updated with property description, accepted values, and the 3.x → 4.x migration warning. Assisted-By: Claude Sonnet 4.6
You're right, naming should be consistent and describe the actual behaviour. Added |
Description
RQueue 4.x switched to Jackson 3.x, which defaults to alphabetical property ordering. RQueue 3.x used Jackson 2.x with declaration order. The same
RqueueMessagetherefore serialises to different bytes depending on which version enqueued it, causing byte-exactZSCORE/ZREMlookups inparkForRetryandackto silently miss — leaving 3.x messages stranded in the processing queue and re-delivered indefinitely via the visibility timeout.Fixes #299
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
ZSETvia a passthrough template, then callsparkForRetryandackusing the live serializer. Using different serializers for write and lookup is intentional — if both sides used the same serializer the test would trivially pass regardless of whether the fix is present.Test Configuration:
Checklist: